Add full clone setting for prebuilds #20177

mustard-mh · 2024-09-04T15:12:07Z

Description

Improve large repository workspace startup performance by fully clone repository in prebuilds phase (download + unzip speed is faster than SCM processing)

Related Issue(s)

Fixes ENT-730

How to test

Add SUPERVISOR_DEBUG_ENABLE=true to preview env's environments https://hw-large-repo.preview.gitpod-dev.com/user/variables so that we get debug logs from supervisor
Create workspace with context https://github.com/JetBrains/intellij-community
- 1-a) Check log of ws-daemon > clone repository took
- 1-c) Check logs of workspace > unshallow of local repository took
Create project for https://github.com/JetBrains/intellij-community, and enable prebuild (only enable), trigger prebuild and create a workspace after prebuild succeed
- 2-a) Check logs of ws-daemon > clone repository took
- 2-b) Check logs of ws-daemon > aria2c download duration, extract tarbal duration
- 2-c) Check logs of workspace > unshallow of local repository took
Enable full clone of project in project settings page, trigger prebuild in UI and create a workspace after prebuild succeed
- 3-a) Check logs of ws-daemon > clone repository took
- 3-b) Check logs of ws-daemon > aria2c download duration, extract tarbal duration
- 3-c) Check logs of workspace > unshallow of local repository took
Duration of 3-b + 3-c should less than 1-a + 1-c and 2-a + 2-b + 2-c
Workspace 3 git functions should works like workspace 1 /2 (i.e. git lg)

✅ Shared test result

Duration		Workspace 1	WS 2	WS 3
		pure workspace	prebuild + workspace	prebuild(full clone) + workspace
a)	ws-daemon git clone	52.920714931s	`prebuild` 49.110767306s	`prebuild` 4m50.046374877s
b)	ws-daemon download and extract tar	NaN	8.946159792s+18.608708639s	26.825172768s+32.283299312s
		`accessable`	`accessable`	`accessable`
c)	supervisor fetch	7m35.843093833s	7m0.284969917s	2.52514ms
summary		`access=53s`, `performance=7m35s`	`access=27.6s`, `performance=7m`	`access=59s`, `performance=0s`

Documentation

Preview status

Gitpod was successfully deployed to your preview environment.

🏷️ Name - hw-large-repo
🔗 URL - hw-large-repo.preview.gitpod-dev.com/workspaces.
📚 Documentation - See our internal documentation for information on how to interact with your preview environment.
📦 Version - hw-large-repo-gha.28542
🗒️ Logs - GCP Logs Explorer

Build Options

Build

/werft with-werft
Run the build with werft instead of GHA
leeway-no-cache
/werft no-test
Run Leeway with --dont-test

Publish

/werft publish-to-npm
/werft publish-to-jb-marketplace

Installer

analytics=segment
with-dedicated-emulation
workspace-feature-flags
Add desired feature flags to the end of the line above, space separated

Preview Environment / Integration Tests

/werft with-local-preview
If enabled this will build install/preview
/werft with-preview
/werft with-large-vm
/werft with-gce-vm
If enabled this will create the environment on GCE infra
/werft preemptible
Saves cost. Untick this only if you're really sure you need a non-preemtible machine.
with-integration-tests=all
Valid options are all, workspace, webapp, ide, jetbrains, vscode, ssh. If enabled, with-preview and with-large-vm will be enabled.
with-monitoring

Dashboard UI Changes

/hold

components/dashboard/src/repositories/detail/prebuilds/PrebuildSettingsForm.tsx

mustard-mh · 2024-09-09T17:41:50Z

✅ Shared test result

Duration		Workspace 1	WS 2	WS 3
		pure workspace	prebuild + workspace	prebuild(full clone) + workspace
a)	ws-daemon git clone	52.920714931s	`prebuild` 49.110767306s	`prebuild` 4m50.046374877s
b)	ws-daemon download and extract tar	NaN	8.946159792s+18.608708639s	26.825172768s+32.283299312s
		`accessable`	`accessable`	`accessable`
c)	supervisor fetch	7m35.843093833s	7m0.284969917s	2.52514ms
summary		`access=53s`, `performance=7m35s`	`access=27.6s`, `performance=7m`	`access=59s`, `performance=1s`

components/content-service-api/initializer.proto

components/content-service-api/typescript/src/initializer_grpc_pb.js

components/content-service/pkg/initializer/git.go

components/content-service/pkg/git/git.go

components/public-api/gitpod/v1/configuration.proto

components/content-service/pkg/git/git.go

mustard-mh · 2024-09-10T08:05:09Z

I will address feedback later

geropl · 2024-09-10T09:14:32Z

Thank you for the nice PR @mustard-mh ! 🙏
I think this is a good solution to the "optimize clone time with prebuilds" usecase. We should definitely roll this out, and if this is helpful for Dynatrace.

One aspect is not covered, though, which is the no-prebuild case: There, we still require the full ~7mins for the "unshallow", where SCM extensions are not usable, because the .git folder is locked - which was the original issue the ticket was created for.

How do you think we could tackle that issue, especially in the context of this change:

would it make sense to add another flag/option to enable an "incremental unshallow" strategy in supervisor right away?
- or should we explore that in a follow-up?
should we create a separate issue for that?

geropl · 2024-09-10T09:53:15Z

Code mostly LGTM, I like this suggestion, the rest if more nits/minor suggestions. ✔️

I tried to test, but somehow my SUPERVISOR_DEBUG_ENABLE env var is not picked up, so I don't get proper log output from the workspace (but also, am trusting you that it's fine). 😄

mustard-mh · 2024-09-10T09:55:32Z

I tried to test, but somehow my SUPERVISOR_DEBUG_ENABLE env var is not picked up, so I don't get proper log output from the workspace (but also, am trusting you that it's fine). 😄

@geropl It takes time to clone, you may need to open workspace and wait for 10 minutes to get the log

mustard-mh · 2024-09-10T10:05:29Z

Thank you for the nice PR @mustard-mh ! 🙏 I think this is a good solution to the "optimize clone time with prebuilds" usecase. We should definitely roll this out, and if this is helpful for Dynatrace.

One aspect is not covered, though, which is the no-prebuild case: There, we still require the full ~7mins for the "unshallow", where SCM extensions are not usable, because the .git folder is locked - which was the original issue the ticket was created for.

How do you think we could tackle that issue, especially in the context of this change:

would it make sense to add another flag/option to enable an "incremental unshallow" strategy in supervisor right away?

or should we explore that in a follow-up?

should we create a separate issue for that?

@geropl I thought we don't need to deal with no-prebuild case

prebuids are used to pre-configure your workspaces to save your time, so full clone is similar to that tasks init (e.g. install npm packages) does: prepare your workspace and make it ready to code
This PR provides a proper workaround for large repository
We tested to fetch with depth does not save time
An incomplete git history can break git functionalities: i.e. I used to do git rebase -i HEAD~10 once workspace startup, and it rebased everything because we clone with depth=1

Co-authored-by: Filip Troníček <[email protected]> Co-authored-by: Gero Posmyk-Leinemann <[email protected]>

geropl · 2024-09-10T12:04:38Z

@geropl I thought we don't need to deal with no-prebuild case

We have, at least in the wider context, because we need to support opening big(ger) repositories, without blocking SCM integrations. Let's tackle that in a separate PR. 👍

mustard-mh · 2024-09-10T22:43:04Z

Double checked result is similar after addressing feedbacks

geropl

Sorry for the delay:

Code LGTM, tested and works! ✔️ 💪

roboquat added do-not-merge/hold do-not-merge/work-in-progress size/S size/M and removed size/S labels Sep 4, 2024

mustard-mh force-pushed the hw/large-repo branch 3 times, most recently from 2ac11f2 to 3148a29 Compare September 9, 2024 06:26

roboquat added size/L and removed size/M labels Sep 9, 2024

filiptronicek reviewed Sep 9, 2024

View reviewed changes

components/dashboard/src/repositories/detail/prebuilds/PrebuildSettingsForm.tsx Outdated Show resolved Hide resolved

mustard-mh changed the title ~~WIP: large repo~~ Add full clone setting for prebuilds Sep 9, 2024

mustard-mh marked this pull request as ready for review September 10, 2024 02:45

mustard-mh requested review from a team as code owners September 10, 2024 02:45

roboquat removed the do-not-merge/work-in-progress label Sep 10, 2024

github-actions bot added team: team-experience team: team-engine team: team-enterprise labels Sep 10, 2024

mustard-mh marked this pull request as draft September 10, 2024 04:11

roboquat added the do-not-merge/work-in-progress label Sep 10, 2024

geropl reviewed Sep 10, 2024

View reviewed changes

components/content-service-api/initializer.proto Outdated Show resolved Hide resolved

geropl reviewed Sep 10, 2024

View reviewed changes

components/content-service-api/typescript/src/initializer_grpc_pb.js Show resolved Hide resolved

geropl reviewed Sep 10, 2024

View reviewed changes

components/content-service/pkg/initializer/git.go Show resolved Hide resolved

geropl reviewed Sep 10, 2024

View reviewed changes

components/content-service/pkg/initializer/git.go Show resolved Hide resolved

filiptronicek reviewed Sep 10, 2024

View reviewed changes

components/content-service/pkg/git/git.go Show resolved Hide resolved

components/public-api/gitpod/v1/configuration.proto Outdated Show resolved Hide resolved

components/content-service/pkg/git/git.go Show resolved Hide resolved

mustard-mh and others added 2 commits September 10, 2024 10:51

proto update

ca2dd7b

Co-authored-by: Filip Troníček <[email protected]> Co-authored-by: Gero Posmyk-Leinemann <[email protected]>

dashboard + server

3af6e9f

mustard-mh force-pushed the hw/large-repo branch from 9d4a9f0 to 3af6e9f Compare September 10, 2024 10:53

mustard-mh marked this pull request as ready for review September 10, 2024 22:41

roboquat removed the do-not-merge/work-in-progress label Sep 10, 2024

geropl approved these changes Sep 11, 2024

View reviewed changes

mustard-mh removed the do-not-merge/hold label Sep 11, 2024

roboquat merged commit 02b4952 into main Sep 11, 2024
19 checks passed

roboquat deleted the hw/large-repo branch September 11, 2024 08:44

Add full clone setting for prebuilds #20177

Add full clone setting for prebuilds #20177

Uh oh!

Conversation

mustard-mh commented Sep 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue(s)

How to test

Documentation

Preview status

Build Options

Uh oh!

Uh oh!

mustard-mh commented Sep 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mustard-mh commented Sep 10, 2024

Uh oh!

geropl commented Sep 10, 2024

Uh oh!

geropl commented Sep 10, 2024

Uh oh!

mustard-mh commented Sep 10, 2024

Uh oh!

mustard-mh commented Sep 10, 2024

Uh oh!

geropl commented Sep 10, 2024

Uh oh!

mustard-mh commented Sep 10, 2024

Uh oh!

geropl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mustard-mh commented Sep 4, 2024 •

edited

Loading

mustard-mh commented Sep 9, 2024 •

edited

Loading